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METHODS AND APPARATUS FOR CONTINGENT TRANSFER AND 
EXECUTION OF SPOKEN LANGUAGE INTERFACES 

Cross Reference To Related Applications « / 7 ^ // 

Sjpdie f\ ^ ^ytAa^e Use. t 
j^jQ^^ The present invention is related to U.S. patent applications entitled "A-ScsteWe- 

Rtsumtc Dialog M ai iaec r ^ai loHiey docket nu. Y0998 520), 'Tersonal Speech 

Assistangr ^tt o mcy docket no."^ Y0998 521) , and **Methods and Apparatus for 

Synchronizing Voice and Text Data in Accordance with Computing Devices" 4 a ttomey 

docket no. YQ99 8 -523 j, all filed concurrently herewith and the disclosures of which are 

incorporated herein by reference. 



Field of the Invention 

The present invention generally relates to spoken language interfaces employed in 
association with computing apparatus and, more particularly, to methods and apparatus 
for contingent transfer and execution of such spoken language interfaces. 

Background of the Invention 

Spoken language interfaces are most commonly encountered in telephony 
interfaces. These interfaces are executed on computing systems which are dedicated to a 
single application such as gathering user data for order taking. In order to accomplish 
this, the user is prompted to utter the required data. Limited command decoding is 
occasionally provided in the form of questions which can be answered with a **y^s'* or a 
'.'no" or a string of numbers. Such an interface is provided for a single application and 
has no requirement for dynamic change in the content of the commands or prompts since 
the dedicated task of the application is imchanging. Further, spoken language services 
are obtained to build the interface by linking an API or application programmers 
int^ace. 

An interface which engages in a dialog with the user, be it free form or carefully 
structured and directed, may be created entirely within an executing application or it may 
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be created for the application by a '^dialog manager." In general, it is desirable that such a 
dialog manager support the initiative of either the user or the application. It is, thus, like 
a communications pipe between the user and the application. Such a dialog manager is 
described in the above-referenced and concurrently filed U.S. patent application 
identified by attomey-xtocgcrn D, YQ998-521 , in the name of L, Coraerford et al., and 
entitled: Personal Speech Assistanj^" It is important to distinguish dialog 
management, as in the above-referenced patent application, fi-om provision of application 
programmer interfaces for the "engines" (such as a spoken command decoding engine) 
providing language related services. APIs, such as Microsoft's Speech Application 
Programmer's Interface and the JAVA consortium JSAPI interfaces, only provide an 
abstraction of the engines' interfaces in order to allow application programs to operate 
regardless of the identity of the provider of the particular engines installed on a given 
user's system. This provides a common low-level interface for accessing the services of 
engines, but leaves the creation and management of dialog to the individual applications 
accessing these low level interfaces. It is a pipe between the application and the engine 
sen^ices, not between the user and the application. 

Also, given the fact that applications and engines can be dynamic in nature, the 
conventional APIs provide insufficient means, or no even no means, for updating 
themselves based on changes in an ^plication or engine. Thus, a need exists for spoken 
language interface methods and apparatus which address these shortcomings in the 
existing art. 



Summary of the Invention 

The present invention provides methods and apparatus for contingent transfer and 
execution of spoken language interfaces. In one embodiment, the present invention 
provides such methodology in the context of a device comprising a Personal Speech 
Assistanta-^P&Ar); in accordance with the above-referenced U.S. patent application 
identified by a ttorney docket no. - Y09Q 8- 53i , and a Personal Digital Assistant (PDA) 
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and application data 1340 (including vocabulary files 1341 and prompt files 1342). The 
PDA 1400 is coupled to the dialog manager 1000 of the PSA via its own serial port 
communications interface 1410. The PDA also comprises an application speech interface 
library 1420, personal information management applications 1430 and PDA operating 
system 1440. 

While other implementations and variations of a dialog manager and user 
interface tables may be employed in the PSA, a preferred implementation is described 
following the detailed description of the PSA in a section entitled: "A Preferred Dialog 
Management System." It is to be appreciated that such a preferred implementation is also 
described in detail in the above-incoiporated conciurently filed U.S. patent application 



id^tified by attofnwTlDClat iiu. Y0998 52Gr in the name of L. Comerford et al, and 
entitled; *^^JSea tebto L u w Rcsumte Dialog Maiiagei " ^ 

As shown in FIG. 2, the hardware implementation of this PSA 2000 embodiment 
comprises a system bus 2010 which couples a central processing imit (CPU) 2020, a 
system memory 2030, a microphone CODEC 2040, a speaker CODEC 2050 and a button 
interface 4150. The PSA 2000 also comprises a serial port 2060 which couples the PSA 
to the serial port 2100 of the PDA 2090, a microphone 2070. a speaker 2080 and 
microphone, volimie and reset buttons 4160. It is to be appreciated that the various 
components of the dialog manager 1000, the spoken language engines 1200 and the user 
interface tables 1300 are executed in accordance with the CPU 2020 and system memory 
2030. The drivers of the hardware engines are also operated in accordance with the CPU 
2020 and system memory 2030, with the hardware components being implemented by the 
corresponding elements shown in FIG. 2. 

More advanced and flexible PSAs enable of providing richer interfaces may 
include multi-ported communications means, nonvolatile memory for recording purposes, 
additional spoken language engines and other service engines such as encryptors and 
decoders in hardware or software form as required for the particular instantiation of the 
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by 3Com Corporation, the MessagePad manufactured by Apple Computer, Inc., and the 
OmniGo manufactured by Hewlett-Packard Company. Thus, as shown in FIG. 6, a PDA 
5500 comprises a screen 5510, a handwriting area 5520, a power button 5530, scrolling 
buttons 5540 and shortcut buttons 5550. 
5 Given the above-described component content of a rich implementation of a PSA, 

a detailed description of its operation in accordance with a conventional PDA will now be 
explained. 

Assume that a user has placed a battery in the battery mounting space of the 
spoken language interface hardware 5000, and placed a conventional PDA 5500, such as 
g 10 a PalmPilot in the PDA mounting space so that the serial connector on the PDA is mated 
with the serial port connector 5020 in the PDA mounting space. The PDA in this 
example has been supplied with personal information management applications 3430. 
These applications 3430 are designed to accept voice commands by creating alias names 
for the graphical user interface names in the standard, non- voice applications. Voice 
15 events are supplied to the application event loop by the application speech interface 3420 

which in turn receives notification of the user's spoken conunands from the serial port 
communications interface 3410, That interface, in turn, is connected to the user's 
conmiand utterances through a chain of physical hardware (FIG. 4) and logical processes 
described below. 

20 Let us trace the creation and issue of a voice event to the communications 

interface to application software of the dialog manager 3000 contained in the PSA 
hardware 4000. 

As described in the above-incorporated concurrently filed U.S. patent application 
identified by attorney d o d c ct no. Y0998 52 0; in the name of L. Comerford et al., and 
25 entitled: "Ar-Sc alablti Low Resourco Dialog Manager " and as is also described below in 
the section entitled "A Preferred Dialog Management System," each application which 
can be addressed by the PSA has a vocabulary file 3341 which contains a table 
comprising three lists. Corresponding elements on the lists comprise a word or phrase 
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